Conventionally, in order to make a human have a sense of affinity for a humanoid robot or an animal-type robot simulating an animal other than a human (hereinafter, simply referred to as a “robot”) which is an actual object, there have been methods attempted to give the robot an appearance similar to an actual human or animal (hereinafter, referred to as a “human or the like”).
Specifically, they include a method to cover the robot with an outer skin formed by a soft material (first method), a method to project a face of a human or the like on the head portion of the robot (second method), a method to coat the surface of the robot by a retroreflector material and project an image of the entire body of a human or the like from an image projecting device using the surface of the robot as a screen (third method), and the like. These methods can make an observer feel as if the robot is a real human or the like, and reduce a sense of strangeness of the robot. The third method is disclosed in “Virtual Reality No Kiso 4, Jinkougenjitukan No Hyouka (the basics of virtual reality 4, evaluation of artificial reality) (supervising editor: Akira Tachi; editor: Toru Ihukube; publisher: Kabushiki-gaisha Baihukan; first edition is published on Feb. 29, 2000).”
However, in the first method, the robot needs to have facial expressions to be close to a real human or the like. To have facial expressions, the robot needs to have many actuators so that the surface of the face of the robot can freely move. Consequently, it results in an increase in the cost of the robot and a complication of controlling the actuators. In addition, since the robot imitates an appearance by having an outer skin, there is a limit that the appearance of only one type of human or the like can be provided.
Further, in the second method, since the face of the human or the like is projected on a display provided on the head portion of the robot, it becomes unnatural unless an observer views the robot from the front. Specifically, when the observer views the robot from the side or back, it can be only seen as an appearance of a robot.
Moreover, in the third method, when there exists some kind of obstacle between the robot and the image projecting device, the shadow of this obstacle will be reflected on the surface of the robot. Therefore, in such a case, it is difficult for the observer to recognize the robot as a real human or the like. Further, the observer cannot even touch the robot because his/her shadow will be reflected when he/she comes close to the robot.
Accordingly, to solve the above problems, a method is conceivable in which a head mount display (hereinafter, referred to as “HMD”) is worn by an observer, and computer graphics in conjunction with movement of a robot are projected on the HMD to be superimposed on the robot.
According to this method, the computer graphics of the human or the like can be freely changed corresponding to the robot. Consequently, facial expressions and postures of the robot can be freely and easily moved. Further, by making the computer graphics of the human or the like as three dimensional computer graphics, the observer will not feel unnatural even when viewing the robot from the side or back. Further, the HMD is a goggle type display which covers eyes of the observer and projects the computer graphics in front of the eyes of the observer. Consequently, the shadow of an obstacle will not be reflected on the HMD. Moreover, the observer can also touch the robot, so that the observer can experience the virtual reality in visual and tactile ways. Therefore, the observer becomes able to feel a much stronger sense of affinity for the robot.
However, the method to project computer graphics on the HMD as described above also has problems. In order to change computer graphics of a human or the like to be projected on the HMD in conjunction with movement of a position or a posture of the robot, it is necessary to calculate space coordinates based on various data detected by sensors on the robot side or HMD side to perform image processing.
When there is a large measurement error in the detected data, or when a time needed for transmitting data or for calculating the space coordinates during image processing becomes equal to or longer than a predetermined time, there will occur a displacement between the movement of the robot and the computer graphics. Further, some kind of disturbance occurring to the robot can cause the robot to move abruptly. Also in this case, a displacement occurs between the robot and the computer graphics.
A displacement occurring by such a reason spoils the feeling of the observer (specifically, the HMD wearer), so that the emotional involvement of the observer with the robot is prevented.
On the other hand, such a displacement between the robot and the computer graphics as described above can be reduced by increasing the accuracy of image processing and the speed of image processing by enhancing the performance of the various sensors, robot and HMD. However, it is impossible to completely eliminate the displacement. Further, mounting high-performance sensors, CPU, and so on in the robot for minimally reducing the displacement leads to an increase in cost, which will be a disadvantage in the economical aspect as well.
The present invention is made to solve such problems as described above, and an object thereof is to provide an image processing system, an image processing device and a display device capable of increasing a sense of affinity for a robot at a low cost.